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LARGE DEVIATIONS FOR THE EMPIRICAL MEASURE OF 
MARKOV RENEWAL PROCESSES 

(N ■ M. MARIANI, Y. SHEN, AND L. ZAMBOTTI 

o, 

CSI , Abstract. A large deviations principle is established for the joint law of the 

^ • empirical measure and the flow measure of a renewal Markov process on a finite 

graph. We do not assume any bound on the arrival times, allowing heavy tailed 
distributions. In particular, the rate functional is in general degenerate (it has 
i^^_^ , a nontrivial set of zeros) and not strictly convex. These features show a behav- 

*^ ' ior highly different from what one may guess with a heuristic Donsker-Varadhan 

analysis of the problem. 

P? 

^ ! 1. Introduction and main results 

-)— » ^ 

1.1. A motivating problem. Consider a finite collection E of servers (or websites) 

in a given network, and a user who visits the servers and downloads data from 

them. It has been observed empirically that the size of downloaded files from the 

^ I web has heavy tails (e.g. a polynomial decay at infinity), and thus a sharp statistical 

O ■ treatment of the matter can be difficult, see e.g. [16]. 

We denote by ipx the probability distribution on ]0, +oo[ of the size of files down- 

lO ■ loaded by a user visiting the server x & E, and by p^^y the probability that a user 

fr^ I visiting x & E will next visit y & E. For optimal hardware or cost /price manage- 

O ' ment, one can then be interested in estimating the laws of various quantities. E.g. 

let us suppose that the cost of downloading a file of size r from the site x is f{x, r) 

and the cost of switching from server x to server y is g{x, y); after downloading an 

, amount t of data, the servers xi, . . . , x^^ G E will have been visited in sequence, 

rS ■ with corresponding amounts of data (ri, . . . , tatJ downloaded respectively on each 

c^ . server. The cost of providing such a service (or the price asked for it) will depend 

on several factors, a reasonable form being 

Nt Nt 

^Tif{Xi,Ti) + ^g{Xi,Xi+i). (1.1) 

i=l i=l 

In the concrete example where the user is a www-crawler, the total time to gather 
data is indeed the "cost" of the service, so that /(a;,r) measures indeed the trans- 
mission performance of server x, while g{x,y) depends on servers lag times. The 
analysis of large deviations quantities (11. ip in the (relevant) limit where the total 
downloaded data are large t — )■ +oo, is exactly equivalent to the large deviations 
of the empirical measures and flow considered below in this paper (however t is 
hereafter interpreted as a time). 

Large deviations principles for processes driven by renewal phenomena have been 
extensively analyzed in the last decade, see e.g. [IHl [HI 1121 [131 US]- In particular 
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one can obtain large deviations principles for the renewal version of processes whose 
detailed asymptotic properties are known, using inversion maps [T9l [TT] . In this 
paper, we consider a case in which these techniques do not work due to the presence 
of heavy tailed distributions of the arrival times of the renewal processes considered. 
To put it shortly, in our setting not only the usual inversion map is not continuous 
(and thus the contraction principle is not allowed), but a naive application of this 
strategy would suggest indeed a wrong result. 

1.2. Main results. Let E he a. finite set, equipped with its discrete topology; ele- 
ments of E are denoted x, y. The set [0, +00] will be equipped with any, compatible, 
complete separable metric (for instance, make it isometric to [0,1]); variables run- 
ning on [0, -|-oo] will be denoted s, t, r. For a separable metric space X, hereafter 
V{X) denotes the set of Borel probability measures on X, equipped with its nar- 
row topology, namely the topology of convergence of bounded continuous functions 
on X. For fj, G V^X) and / a measurable, /i-integrable functions on X, fi{f) de- 
notes the integral of / w.r.t. /i. For instance for ip G 'P([0,+oo]) we understand 
4'{e'^) = J Tpi^dr) e^ . If /x, z/ G V{X), H(z/ | /i) denotes the relative entropy of v w.r.t. 
/i: 

H(z/ 1 /i) := sup z/((y9) — logyu(e'^) 

Let E X E 3 {x,y) ^ Px,y ^ [0, 1] be an irreducible Markov kernel on E, and for 
X G E let ipx G P([0, +00]). We do not assume any moment bound on ip^, but only 
i^xii^}) = "ipxii+oo}) = for all x & E. Fix an arbitrary initial measure 7 G V^E), 
and consider the Markov renewal process {Xk,Tk+i)k>o where 

• (Xfc)fc>o is an irreducible Markov chain on E with probability transition 
matrix {Px,y)x,yeE, and initial distribution 7 (for Xq). 

• (tj)j>i is a random sequence in ]0, -|-oo[, such that conditionally to (X^) the 
Ti are independent and have distribution 

F{n G A I (Xfc)fc>o) = ^x,_i(A), A C]0, +oo[ 

We refer to [H VII. 4] for the presentation of the Markov renewal processes framework 
introduced above. Note that often one assumes the law of the Tj conditioned to 
{Xk)k>Q to depend both on Xj_i and Xj (namely, one may replaced ipx.^i with 
a 4'Xi.i,Xi above). However, one can always reduce to the case here described by 
considering the Markov chain (Yk)k>o, Yk '■= {Xk,Xk+i) instead. As we have no 
hypotheses on ip^, this doubling- variables procedure is fully compatible with our 
framework. 

For t > and n > an integer, define the switching times Sn and the number Nt 
of switches up to time t as 

n 00 

Sn ■= ^n, n> 1, Nt := ^ l(s„<t) = inf {ri > : Sn+i > t}, 

1=1 n=l 

where we understand empty sums to vanish and inf = +00. 
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Fixed t > 0, we then define the empirical measure ^t ^ V{E x [0, +00]) of 
(Xat^, ttvJ up to time t by requiring for all / G Cb{E x [0, +00]) 

M)-=-i:l f{XN,,TN,+l)ds 



'0 

Nt 



k=l 



:i.2) 



Similarly we define the empirical fiow Qt G C{E x E\ [0, +00 [) as 

Nt+l 
t 



. Nt+l 

Qt{x,y):=-J2'^(^k-i=x,x,=y) x,yeE (1.3) 

fc=i 

Note that the process {Zt := XivJt>o has a natural interpretation. Pick an initial 
datum Zq = Xq with law 7 on E. Then pick a time ri > with law ipxo^ and at 
time Ti Zt jumps to Xi, picked on E with distribution pxo,-- Now Z spends a time 
T2 (with law ■i/'Xi) at Xi, thus jumping at X2 at time ri + T2 and so on. Thus, fj,t is 
the joint empirical law of the process Zt and the associated inter-jumps times, while 
Qt{x,y) is the total number of jumps of Zt from x to y up to time t. 

We want to investigate the large deviations of the joint law of {fit, Qt) under the 
probability P^ of the renewal Markov chain {Xk,Tk)k>Q. Before stating our large 
deviations result, we need some further definition to introduce the spaces and the 
rate functional. 

Let A := V{E x [0, +00]) x C{E x E; [0, +oo[), which is a completely metrizable, 
separable topological space under the narrow topology of V{E x [0, +c>o]) and the 
uniform topology of C{E x E; [0, +oo[). Then define ^ G C{E; [0, +00]), and Aq C A 
as 

«.):^.up{e>0 :/«..)." <+.o} 

Ao:= i(/i,g)G A : / ^(x,rfr)- = XQ(x,i/) = J]g(|/,x), Va;Gsl (1.4) 
I -^[0'+-l ^ y€E yeE J 

For (/i, Q) G Aq define the Markov transition kernel p*^ and the map E 3 x -^ ipt^ E 
P([0,+oo]) as 

■^xidr) ■■= ^ 7v 7-Kx,dT) xeE 

Y.z&eQ{^^z) ^ 
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where, if Q(x, ■) = 0, the (irrelevant) choice p2,y = Px,yi V'x — V'x is understood. 
Then define I: A — )■ [0, +00] as 



i(/^,Q) = < 



if (/i, Q) e Ao 



+ Ua;)l{oo}(r) 



+CXD otherwise. 

(1.6) 
where hereafter we adopt the convention • 00 = 0. A variational carachterization 
of I is given in Proposition 12.21 

Proposition 1.1. The functional \ is good, namely for each M > the set {(/x, Q) : 
I(/^5 Q) < M} is compact. 

The following theorem is the main result of this paper. 

Theorem 1.2. The /aw;P^o(/ij, Q^) of {fit, Qt) underF^ satisfies a large deviations 
principle as t ^ +00, with speed t and good rate I, uniformly in'j E 'P{E). Namely, 
for each closed set C G A and each open set O G A 

limsup- sup log P^((/it, Qi) gC) < - inf I(/i, Q) (1.7) 

limsup- inf logP^((/iT, Qt) e C*) > - inf I(/i, Q) (1.8) 

1.3. A contraction principle. From the previous result, let us project I on the 
first component. Namely, 

Ii(/i) = inf{I(/i,Q), (/i,Q)GA}. (1.9) 

Then, by the contraction theorem, the following result follows immediately 

Corollary 1.3. The family {P^ o IJ^^,T > 0}, with respect to x E E, satisfies a 
large deviations principle with good rate Ii defined by U.9\) as T ^ +00 with speed 
T, i.e. for each closed set C G V{X) 

limsup- log P,.(;UT G C) < - inf Ii(/i) (1.10) 

T^+00 J i^^(^ 

and for each open set O G V{X) 

liminf ^ logP,(/iT eO)>- inf h{fi) (1.11) 

We give an expression for Ii in Proposition 17.11 below. 

The paper is organized as follows. The necessary notation and definitions are 
introduced in section 11.21 together with the main results. In section 11.31 we briefiy 
describe some consequences of these results. In section [2] the properties of the 
rate functional are discussed, and in sections HHS] the proofs of the large deviations 
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principle are provided together with further remarks. In section [7] the other results 
are proved. 

2. The functional I 

In this section some " deterministic" results concerning the functional I are estab- 
lished, and Proposition 11.11 is proved. 

For if e Cb{Ex [0, +CX)]), {x, r) ^ Vx{t), c, M e C{E) such that < c(x) < ^(x), 
with c{x) < ^{x) for all x such that ^(x) > and M{x) > define h"^'"'^ : E x 
[0,+oo] I-)- [0, +oo[ as 

;,^,c,M^^)^^^^^^^^j^^^^^^^^^^(^)^ ^^^^ ^e]0,+oo], (2.1) 

for any choice of ip^ ^ C'blJO, +oo]), M^ > 0, c^, < ^^ if Cx > otherwise c^ := 0. 
Notice that /i^''^'*^ is l.s.c. on ]0, +oo]. Then we define 

V:=\{h,H): h = /i'^'^'*^ H -.ExE^^, 

(2.2) 



7/>, {e^^^) < 1, ^g.,.e^(^'^) < 1, Vx G i^l. 



The following lemma is used to provide a key carachterization of I in Proposi- 
tion O 

Lemma 2.1. For all a > and x, ^ G V{[0, +C)o[) such that J xid^) ~ < +oo 
Xidr) i H(^ I V') = sup(7r(/i/r) - 7r(l/r) logV'(e'^)) 

h 

= sup (7r(/i/r) — 7r(l/r) log'?/'(e'^)) = sup 7r(/i/r) 

h:il)(e'^)=a h:ip{e'^)=l 

where suprema are taken over h G Cf,(M!i_). 
Proof. It is well known that 

H(^|^)= sup {n{h)-\og^{e'')). 

heCtitLl) 

Now, suppose that ipi^e^) = a > and set h"- := h — log a. Then 

7r(Vr) - 7r(l/r) logV'(e'^) = ti^H^/t) - 7r(l/r) logV^(e'*'') 
and iple^") = 1. Therefore the quantity 

sup (7r(/i/r) - 7r(l/r) logV'(e'')) = sup (7r(/i/r) - 7r(l/r) log^(e'')) 

h:ip{e'^)=a /i:V'(e^)=l 

does not depend on a > and thus 

sup (7r(/i/r) — 7r(l/r) log'?/;(e'*)) = sup 7r(/i/r) = 7r(l/r) H(7r | ■?/') 

where all suprema are taken over h G Cb(]Rl). D 
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For all {h, H) G F we denote by Ih,H : A i— )■ M the functional 

x,yGE \ z / 

+ E 1 / f^{x,dT)K{T)-J2Q{x,y)\og f Mds)e''^^'^ 

^ V-']0,+oo] „ ^10,+oo[ 



(2.3) 



y 
Proposition 2.2. For all {jj, Q) G U 

I(/i,Q) = sup lh,H{f^,Q)- (2.4) 

{h,H)er 

In particular, I is convex on A. 

Proof. Fet first {fi, Q) G Uq and let us consider [h, H) G F. Then, by the well 
known properties of the relative entropy and since ^x > Cx, we find easily that 
I(Ai, Q) > ^h,H{l^, Q), so that by the arbitrariness of {h, H) eT 

I(/i, <5) > sup lh,H{fJ',Q)- 
{h,H)£r 

Now, let us prove the converse inequality. We have for all {h, H) G F with h = h'^^'^^^ 
sup sup lim sup (/i, K^''^'^^) 

if c A/— 5>+oo 

= sup ^ (;u(x, 1/r) /i(x, v?:r -logV'^(e'^")) + fi{x,+oo) ^x) 

^ X 

= sup ^ (/i(x, 1/r) /i(x, ipx) + /i(x, +oo) ^x) 
= ^ {n{x, 1/r) H (/i(x, ■) I ipx) + Kx, +oo) ^x) ■ 

X 

On the other hand 

sup {Q,H) = y2(y2 Qi^^ y) ) h (g(x, ■) i g.,) . 

Therefore 

sup Ih,H{fJ',Q) > I(yU,<5)- 

{h,H)er 

Finally, let {fJ,,Q) G U\Uo; we want then to show that sup(^f^jj^^Y'^h,H{fJ^,Q) = 
+00. Since (/x, Q) G U \ Uq, then /x(x, 1/r) < ^ <5(a;, y) for some x E E. Then for 
if (x, ■) = and hx{T) = —M/r we have 



hMf^, Q) > M ( ^ Q(x, y) - /i(x, 1/r) j 



-> +CX). 



We turn now to show the proof of Proposition II. 1[ 
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Proof of Proposition \l.l[ Let (yU„, Q„) C 3^ be a sequence of measures such that 

hm I(/i„,Q„) < +00 (2.5) 

We need to prove that (//„, Q„)„ is compact (coercivity of I) and that for any hmit 
point (/i, Q) of {fin,Qn), hminf„^+oo I(/Wn, Qn) > I(a*)Q) (lower semi-continuity of 
I). Notice that (12. 5p imphes (/in,Qn) C Uq for n large enough, i.e. 

Hn{x,l/T) = ^Qn{x,y), ^iQnix,y)-Qniy,x)) = 0, \/xeE. (2.6) 

j/eE y&E 

Coercivity of I. By the bound (12. 5 p 

lim /i„(x, 1/r) H(/i„(x, ■) | ^^) < +00. (2.7) 

From any subsequence, we can extract a sub-subsequence along which 

H(/2„(a;, ■) | il)^) -^ 6, /i„(a;, 1/r) -> a, 

with a, 6 G [0, +00]. Let us set for simplicity := ip^ and 7r„ := /i,i(x, ■). If 6 = +00, 
then a = 0, thus let us suppose that h < +00; then by the inequahty tlogt > — e^^ 
and by Jensen's inequality we get 

-0(R; \ E) + H (7f„ I 0) > / /„ In /„ c?0 > ^„(E) log ^^, (2.8) 

e Jb 0(^) 

for any Borel E d M., where 7f„ = /„0. Choosing _E =]0,e[ with < e < +00, 
we find that sup„7r„(i5) — )■ as e — )■ 0. Therefore (vr„)n is tight in P(]0, +00]). 
Therefore, up to passing to a further subsequence, 7f„ ^^ vf G P(]0, +00]). By lower 
semicontinuity of the relative entropy H ( • | 0), we obtain that H (vf | </>) < +00 and 
therefore vf ({+00}) = 0({+oo}) = 0. We claim also that 7r„(l/r) — > 7r(l/r) < +00. 
Notice that 

- -Knidr) = 7r„(l/r) 7f„(]0, e[) 
and, by boundedness of (vr„(l/r))„ and tightness of (vr„)„ we obtain that 



f 1 

lim sup / — 7r„(dr) = 0. 

£^0 n J]o.e\ T 



Therefore, by uniform integrability, we obtain that 

7r(l/r) = lim7r„(l/r) < +00. 

n 

It follows that necessarily a < +00. Let us denote 
Km 



(/i, g) G A : ^(1/r) < Y, Qi^' y)<M\ 

x,y J 



Since {jJn^Qn) G Uq, then supj,Qn{x,E) = sup„/i(x, 1/r) < +00, then (/i„,Q„) G 
Km for M large enough. It is not difficult to verify the compactness of Km for all 
M < +00, therefore we can conclude compactness of (/i„, Q„)„ in 3^. 
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Semi- continuity of I. Let {fin,Qn)n C A be such that {fin,Qn) — > (a*, <5) in A. We 
want to show that 

I(At,<5) < hml(/i„,(5„). 

n 

Since U is closed in A and I = +00 on A \ U, we can suppose that (/i, Q) G U. 

Let us consider {h,H) G F, see fl2.2p . Since h is s.l.c. and U is closed in A, then 
Ih,H is also l.s.c. on A. Then by f l2.4p we obtain 

h,H{fJ', Q) < liniIfe,H(yUn, Qn) < liml(/i„, Q„) 

n n 

and by the arbitrariness of {h, H) 

l{fi,Q)= sup I/j,/^(/i,Q) < limI(/i„,Q„). 

{h,H)er n 



n 



Lemma 2.3. Let U 6e defined by f lL4p . anc? /ei 
Uoo := { (/i, Q) e U : I(/i, Q) < +00, 

li{x, +CX)) = and ij,{x, ]0, +oo[) > 0, Vx G -E, (2.9) 

{Q{x,y))x,y^E defines an irreducible transition matrix on E}, 

where Q is defined as in i \1.5\} . 

Then Uoo is 1-dense in U , namely for all (^, Q) G U with I(/i, Q) < +00, there 
exists a sequence {fin, Qn)n in Uoo such that 

(/in,Qn) -^ (/i, <5), liml(/i„,(5„) = I(/i,Q). 

n 

Proof. Let us start by proving that the following set 

Ui := { (/i, Q) G U : I(/i, Q) < +00, /i(a;, ]0, +oo[) > 0, ^x e E, 

{Q{x,y))x,yeE defines an irreducible transition matrix on E}, 

is I-dense in U. For any x G -E, let A^ be a bounded Borel subset of ]0, +00 [ such 
that tpx{Ax) > 0. We can set 

fi[x,dT):= , Z ■.= 2_^Vy Ti)y[dT\Ay), Q''{x,y):= ^^, 

yeE -^ 

where Z < +00 since every Ay is assumed to be bounded. Since {qxy)x,yeE is 
assumed to be irreducible, Q^ is so as well; at the same time, it is easy to see 
that I(/i°,Q°) < +00 and therefore (/i°,Q°) G Ui. We note in particular that 
/i°(x, ■) ^ i'xi') and Q^{x,-) <^ qx- for all x E E. Now for any {fi,Q) G U with 
I(/i, Q) < +00, we set 

/x":=e/i +(l-e)/i°, g":=£g + (l-£)g°, eG[0,l]. 

Then (/i*^, Q"^) G Uq and by convexity 

I(/i^Q^)<£I(/x,g) + (l-£)I(/i^g°) 

and we obtain that (yU^, Q^) — t- (yU, Q) and ^/i"^, Q^) — ?■ I(/i, Q) as e — )• 1. 
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Let US now show that Uoo is I-dense in Ui. We note first that 

lhK-J-logV'x([M,+oo[) = -ex, VxgE. (2.10) 

M JVI 

Indeed, by the exponential Markov inequahty, for all < c < ^ (we drop the 
subscript x for simplicity of notation) 

7/'([M, +00 [) < ^(e"("-*'^)), VM > 0, 

so that, by taking the supremum over c < ^. 

li^ — logV'([M,+oo[)<-e 

Now, if ^ < +00, suppose that the inequality is strict, i.e. there exists c > C, and 
Mc large enough such that 

V'([M, +oo[) < e""^, ^M>Mr^. 

Then we have for all d < c 

^(e^'O = / tpie"'^ > t) dt < I + c' ^{t> s)e''''ds 
Jo Jo 

r+00 

< 1 + e''^^^ + / c'e-("-"')^ ds < +00. 

J Mc 

In this way, the definition of ^ is contradicted and we have proved ( ]2.10p . Therefore 
there exists a sequence (M„(a;))„gN such that Mn{x) — )■ +00 and 

^^"^ nj ( ^ log^x([Mn(x),+oo[) = -^^, 

n Mn[x) 

Let us now fix {ji, Q) G Ui and x & E. If yu(x, +00) = 0, then /i„(x, ■) := /i(x, ■), 
Qn{x,-) := Q{x,-). If /i(x, +00) > 0, then, since I(/i, Q) < +00, necessarily ^^ < 
+00 and therefore ^/'^^([M, +00 [) > for all M > 0. Moreover, since (/i, Q) G Ui we 
have /i(x, Ir) > and finite; finally, yu(x, ■) must be of the form 

fi^Xjdr) = p{T)ipx{dT) + ad^ooidr). 

Now, denoting J„ := [M„(x), +oo[, we set for a„, /3„ > 

/i„(x, ■) := (a„p(r) + /^n Q , /" ) ^xidr). 

Now we want to fix a;„, /3„ such that 

/i„(x, ]0, +oo[) = /i(x, ]0, +00]), /i„(x, 1/r) = fi{x, 1/r), 

i.e. a„ yu(x, ]0, +oo[) + /3„ a = /i(x, ]0, +oo[) + a and 

a„/x(x, 1/r) + /3„aV'^(l/r I J„) = /i(x, 1/r). 

This linear system has a unique solution («„, /3„) if 

/i(x, ]0, +cx)[) V^^(l/r I /„) - /i(x, 1/r) 7^ 0, 

which is true for n large enough, since ipxi'^/T \ [M, +C)o[) — )• as M — ;■ +00. For 
the same reason, a„ — )■ 1 and /3„ — )■ 1. In particular, this shows that yU„ ^ /i on 
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£^x]0, +00] as 77, — )• +C)o. Setting Q„ := Q, since /i„(x, 1/r) = fi{x, 1/r) for all x E E 
by construction, we have that (//„, Q„) G Uoo- 

Notice now that, since /i„(x, ]0, +C)o[) = fi(x, ]0, +00]), 

/i„(x, ■) = a„/i(a;, ■) + (!- a„) i/„(-), 

where 

z/„(rfr) = - ,../%. i^xidr). 

T 'ip^{l/T;In) 

Now, by the convexity of H(- | ipx), 

H(/i„(x, ■) I ij^) < a„ H(/i(x, ■) I ij^) + (1 - a„) H(z/„ | ^p^). 
Now, recalhng that /„ = [M„(x), +00 [, 

[l-an) H(z/„ I ^^) = I3na — - — —-- log 



/i(x, 1/r) \^xiln] 

~ /i(a;, 1/r) M„(x) VV^^^l/n)/ /i(x, 1/r) 
as n — )■ +00. Then, it follows easily that 

limsupl(/i„,(5n) < Kf^,Q)- 

n 

Since I is lower semi-continuous, the proof is complete. D 

3. Proof of the large deviations principle 

We denote X := ii^x]0,+oo]. We recall the expression (11 ■2p of the empirical 
measure fit of the process (^at^, r7v^+i)s>o for all bounded continuous function / : 



1 ^' + _ Q 



t ^^ ^ ^ ' ' t 

k=l 



where an empty sum is defined as zero. 

Our aim in this paper is to study the large deviations of the empirical measure fit- 
We here extend the large deviation principle for fit by considering also the empirical 
flow, namely the process 



. Nt+l 

Qt{x,y) -=-^2 Mxk-i=x,x,=y) x,yeE, t > 



fc=i 



which has already been defined in (11.31) . Namely, Qt{x, y) is the number of transitions 

of {Xk)k=o,...,Nt+i fro"^ ^ to y- 

The main result of this paper, in Theorem II. 2l below. is a large deviations principle 
for the law of {fif,Qt) under P^, as t — )■ +00 with an explicit rate function I defined 
in Definition 11.61 This allows to deduce the large deviations principle for the law of 
fit described in Corollary 11.31 
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4. Upper bound 
We define 

U := J (/i, Q) G A : fi{x, 1/r) < ^ Q(x, y), 5^(Q(x, z) - Q{z, x)) = 0, Vx G E i 
I y z ) 

Uo := ] (/i, g) G U : /i(x,l/r) = J]g(x,y),VxG^i. 
I yeE J 

Lemma 4.1. Let O C A be open and C D Aq. Then 

limsupilogP^((/it,Qi) ^0)=-oo. (4.1) 

i— i>+oo 

Proof. U is closed by Fatou's lemma. By tlie definitions (11. 2p and (II. 3p . we have 

Nt 



V^) = 7£l(x.-i=-) + 



Namely 



Moreover 



t — SNt 

. Nt+l 
y&E k=l 



X](<3t(a;,y) -Qt(y,x)) 



l(Xo=xO - l(X^r , i=aO 



1 
< -. 

- t 



t 

Then (14. ip follows, since {^^ Qt) ^ V for t large enough. D 

4.1. Exponential tightness. In order to prove the upper bound we have to show 
the exponential tightness of the sequence {P o (/i-r, Qt)~^}t>o- To do so, we need 
to show that 

inf lim - logY* AlC) = —oo. 

K t->+oo t 

where /C varies among all compact subsets of A. 

Lemma 4.2. Setting Km := |(/i,Q) G A : /i(l/r) < Ex,y<5(^'^) ^ ^} ^^^^ ^m 
is compact in A and 

lim lEK - logP((/it, Qt) G KIj) = -cx). 

M— J>+oo t-J-+oo t 

Proof. We recall that ;Ui( 1/r) < ^2^ QAx. y). see the proof of Lemma 14. II Moreover 

Nt 1 t-Sr,, . Nt + l 
f^t{l/r) = — + - < — 7 — , 
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SO that {/it(l/r) > M} C {A^t + 1 > n} = {5„ < t} for n = [Mt\ . Then 

{{fit, Qt) e Kl,} = {fitil/r) > M} = {N, + l>Mt + l} = {S^Mt+n < t}. 
By the Markov inequahty, 

nif^uQt) e Kl,) < nS[Mt+ii <t)< E(e*-^LM*+ij) < e*+LMt+ijiogc 
where c := supy^^ipyi^'^) < 1- Then we have 

hmsup - logP((/ii, Qt) G Kl,) < 1 + (M + 1) logc 

i— >+oo i 

and this tends to — oo as M — )■ +oo. Compactness of Km is standard. D 

4.2. Change of probability. To prove the upper bound, fix {h, H) G F. We define 

and we call P^'*'-^) the law of the renewal Markov process (Xk, Tk+i)k>o with transition 
probability {q^^ip^). Then, 

1 , dF^^^"^ 
log 



: J] Q,(x, y) (h{x, y) - log J] g.,.e^(^'^) j 
+ V ( / /ii(x,dr)/i,(r)- VQ,(a;,y)log / V-.l^s) e^'^^^ 

^ \^]0,+oo] ^ ^10,+oof 

, TAft+1 -t+ SNt , , ^ 



TAff+l - t + 5, 



A^t 



— I/i,H(/^i,Qi) H ^ 7 ^ ^XjVj(TArt+l), 

where Ih^u is defined in (12. 3 p above. Now, recall that h takes the form (12. ip . Then 
on the event {n = Nt^ 1, X„_i = a;} 



^iVt + l - i + 5'Af r„-t + S'„_i /v5a;(T-n) / \^^ HV^I 

-/^Xjv,(l"iVt+l) = 7 ^ C^l]M,,+oo](7-n) > 



, - y\-j\^ \ isT^r -I- / I I '-J, — jjv^XjT^'-*^] V "■/ I ( 



OO 
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since < r„ — t + 5'„_i < r„ and c^ > 0. Therefore, for A measurable subset of A 
and for {h, H)eT 



1 1 / . ^ M „ dF^^'^'^ 
-logPt(^) < -logE l^(/ii,Qi)e""'^'^^''"'^*^+"^"°° 



t ^ '' ' - t ° \ ^^^^'^'^ dF 



0'i(Xk,'''k)k<Nt + l) y 



< -log 

- t ^ 



jm{h,H) 

g-*inf(p,Q)6^Ih,H-(M,Q)+ll'^l|oo ]g ' 



dF 



<^{{Xk,'''k)k<Nt + l) J 



and therefore 

mv -\ogVt{A) < - inf I/,,^(/i,Q). (4.3) 

t->+oo t (m,Q)G^ 

For M > 0, ^ G Cc(]0, +oo]) for all a; G ^, G : ^^ h^ M and 5 > 0, let 
KM,g,G,6:={{li,Q)eKM : 3{ii' ,Q') e\5Mg) - ii\g)\ + \Q{G) - Q\G)\< 5}, 



where Km is the compact set defined in Lemma [4.21 and 

RM,g,G,5 ■■= - lim -logPt(ii'^ G,5)- 

Let now O be an open subset of A. Then applying (14.31) for ^ = C fl KM,g,G,( 
lEK ^logPi(0)< Ih^ l\og[2ma.x{Pt{On KM,g,G,s),Pt{KligG5))] 



< max I — 

{^l,Q)&or\KM 



inf h,H{P-, Q), —RM,g,G,5 1 

9n-ft'M,9,G,<5 / 



inf lh,H{fJ',Q) ^RM,g,G,S 



which can be restated as 



hm ^logP,(0) < - inf I,,^,M,5,G,5(/i,Q) (4.4) 

for any open set O, where the functional lh,H,M,g,G,s is defined as 

Ih,H(/i, Q) A RM,g,G,S if (/i, Q) e KM,g,G,5 



^h,H,M,g,G,s{f^^ Q) '■' 



+00 otherwise. 



Since h is Isc and KM,g,G,5 is closed, then lh,H,M,g,G,5 is Isc. By minimizing (14. 4p over 
{/i, if, M, g, G, 6} we obtain 

lim -logPt(C)<- sup inf lh,H,M,g,GAl^) 



Since O is arbitrary, by applying the minimax lemma [141 Appendix 2.3, Lemma 

3.3], we get that for any compact set /C 

lim -logPt(/C)<- inf sup lh,H,M,g,G,5{l^^Q) 

*^+°° t {/i,Q)e/C h,H,M,g,G,6 
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i.e. (Pt)f>o satisfies a large deviations upper bound on compact sets with speed t 
and rate functional I : A h-)- [0, +00] given by 

i{fi,Q):= sup lh,H,M,9,GAf^yQ)- 

h,H,M,g,G,5 

By Lemma WT\ we have rig^G,5KM,g,G,s C U, so that I(/i, Q) = +C)0 if (/x, Q) ^ U. We 
claim now that 

lim RM,g,G,5 = +00, yg,G,5. 

Indeed, by the definition we see that Km fl U C KM,g,G,s- Therefore Pt(-f^M,c/,G,<5) — 
2m.a.x{Pt{Kl.^),Pt(\J^)}, and we conclude using (14.11) first and then Lemma l42l 
Therefore for all (/i, Q) e A 

Kfj',Q) > sup ih,H{f^,Q) 

where 

(lh,H{fi,Q) if(/i,Q)eU 
lh,H{fj^,Q) ■■= I 

1+00 otherwise 

Thus I(/i, Q) > I(/i, Q) by Proposition 12.21 Therefore (Pi)t>o satisfies a large de- 
viations upper bound with rate I on compact sets. By Lemma 14.21 and [3, Lemma 
1.2.18], (Pi)t>o satisfies the full large deviations upper bound on closed sets. 

5. Lower bound 

For the proof of the lower bound, let us denote by P^ the law of {fit, Qt)- Then it 
is well known that it is enough to show the following 

Proposition 5.1. For every (/i, Q) ^ Uo ('•''^d t > 0, there exists a family of proba- 
bility measures Qt such that Qt ^^ 5{^l,Q) as tl; +00 and 

limsup^H(Qt|Pt)<I(/i,Q). 

Indeed, if Proposition 15.11 is proved, then we reason as follows. Let (//, Q) G Uq 
and let V be an open neighborhood of (yU, Q) in the weak topology. Then 

log P((„. 0,) e V) = log /^ ^dQ. = log (^ /^ ;|idQ,) + log Q,(V) 



> — 

- Qt 



m/v^°^S'^* + ^°^^'^^^- 



by using Jensen's inequality. Now, since xlogx > — e ^ for all x > 0, we obtain 
> qTyy (-H(Q, I P.) - e-') + log Q.(V). 
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Since {fi,Q) G Uq, Qi ^ ^(m.q) and V is open, then Qf(V) — t- 1 as t — )• +00. We 
obtain 

liminf ^logPi(V) > -limsup ^QilPt) > -l{f^,Q). 

t^+00 t t->-+oo t 



Therefore, for any open set O and for any (/i, Q) G U, 



0, 



hminf^logPi(0)>-I(/i,Q). 

and by optimizing over (/i, Q) & O we have the lower bound. 

Proof of Proposition 15. 1[ Let us first suppose that (/i, Q) G Uoo as defined in f l2.9jl . 
i.e. (/i, Q) e Uo, /i(x, +CX)) = 0, 

Z, :=/i(x,l/r) = J]Q(a;,z) >0, Vx G E, 

Q{x,y) := Q{x,y)/Zx defines an irreducible probability transition matrix on E and 
fi{x, ]0, +cxd[) > for all x E E, so that /i(x, 1/r) > for all x E E. Moreover, since 
(/i, Q) G Uo, then /i(x, 1/r) < +00 for all a; G -E and in particular fi{x, ■) ^ 'ipxi') 
and Q(x, ■) <^ g^,- Then it makes sense to define for x,y E E 

In this way, the probability kernels q^ and ip^ defined in (14. 2 p become 

X 11 

g^ = — Q(x, y) = Q(x, y), V'x(^^) = ^ - /^(a;. dr). 

^x ^x T 

We denote by Px ' the law of a Markov renewal process (X^, rfc+i)jt>o with 
transition kernel (g"^, '0'') and initial state Xq = x. By the irreducibility of g^ = Q, 
the unique invariant measure v^ of g^ is given by 

j/GE 

since we have 

Moreover, since /i(|/, +00) = for all y G -E by the assumption (yU, Q) G t/00, 

Then, we have by Proposition 16. II below that P^ ' -a.s. 

fit{y,dT)^ Zu^Tilj'^{dT) = fi{y,dT), Qt{y,z) ^ Z u^ q^^^ = Q{y, z) 

as t — > +00. We set Tt := [(1 + 6)Zt\ and we denote by Pj,5 the law of (X^, Tk+i)k>o 
under which 
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(1) (Xfc, Tfc+i)/fc=o,...,rt is a Renewal Markov process with transition rates (qjip) 
and Xq = X a.s. 

(2) conditionally on {Xk,Tk+i)k=o,...,Tt, {Xk,Tk+i)k>Tt is a Renewal Markov pro- 
cess with transition rates {q,ip). 

Then we denote by Qt,s the law of (/if, Qt) under Ft^s. Let us prove first that 

lim lim Qt,5 = '5{f.,Q). (5.1) 

o4,U ij+oo 
{h,H) 



By Lemma [6. 3[ under P^. ' we have a.s. 

hm ^= hm ^^ = E^)i''\n)Zil + 5) = l + 6. 

However St^ has the same law under Pi. ' and under Ft^s, so for any 5 > 

lim FtsiSr, < t) = lim P^'^'^^ ( ^ < 1 ) = 0. (5.2) 

Therefore, if we set 

A,5 := {St, > t} 
then, by (15. 2p we obtain that for all 5 > 

lim Pf,5 {Dt,s) = 1. (5.3) 

t^ + OD 

We recall that {Sn > t} = {Nt + 1 < n}. Therefore on Dt^s we have Nt + 1 < Tt 
and setting for any / G Ch{X) and e > 



M,e ■= i l/^t(/) -/"(/) I >^. sup |Qt(x,i/) -Q(x,y)| > e 

I x,y&E 

we have 

Ft,s{Aie) < ri''''\Al, n A,.) + ^tADls). 
By Proposition 16.11 

hm P^^) f |/x,(/) - /x(/)| > e, sup |g,(x,y) - Qix,y)\ >e)=0, 

which, in view of (15. Sp . implies (15. ip . Now we estimate the relative entropy 

H(Qi,, I P,) < H(P,,5 I P.) = tEt^Jt^s) = tE^t''\Jt,&) 
where 



j=i 
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By the ergodic theorem, we have 

Tt-l 



hm Ei^'^) 1 Yl (HiX.-uX,) + r, /ix,_,(r.)) 



t^+oo -^ \ t 

i=l 



1 + 6 
Z 



J2''^\Y1 "ily ^(^' ^) + ^'(^ ^^ 



since 



;i + 5) ( E ^(^' ^) ^(^' ^) + E '"(^' ^-) ) < (1 + ^) i(^> Q) 

\ x,y X / 



Therefore 

hmlh^)H(Q,,5|P,)<I(/i,Q). 

510 t^+oo t 

Then there exists a map t h-> 5(t) > vanishing as t f +oo such that Q^ := QtMt) ~^ 
5(^^,Q) and h^t t-^ H(Qt | P*) < I(/i, Q). 

D 

6. Law of large numbers 

In this section we want to prove the following law of large numbers for the empiri- 
cal measure and flow under the additional assumption that all probability measures 
ipy, y & E, have finite first moment. 

Proposition 6.1. Suppose that 

^An) = 'Y'^yi'yi^) < +00. 

y 
Then, for all x,y,z & E, under F^-a.s. 

lim— ^E^(ri), lim /ij(|/, rfr) = — -^ rV^y(rfr), lim Qt{y,z) = J '^\ . 

For any y E E we denote 

0? := inf{£ > : X,_i = y], 0^+, := inf{£ > 0| : X,_i = y], k> 1. 

Moreover we define 



fc=i fc=i 

i.e. the number of times the process {Xk)k=o,...,Nt-i visits the site y, and 

n—l +00 

My := Y Mx,=y) = E l(^r<")' "^y ^ ^- 

fc=0 fc=l 

i.e. the number of times the process {Xk)k=o,...,n-i visits the site y. 
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Lemma 6.2. For any x,y G E, under F^ the sequence {T^y)k>i is a i.i.d. sequence 
with common distribution ipyi-). 

Proof. Setting .^^ := a{Xk, A; > 0), we have that {4>\)k>i is ^^-measurable. Now, 
conditionally on ^^, the sequence (tj)j>i is independent. But conditionally on 
^"^, T^y has law ipy{-). Therefore, the conditional law of (T^!')fc>i given ^"^ is the 
law of a i.i.d. sequence with common distribution ipy{-). Since this conditional law 
does not depend on (X^, k > 0), the result is proved. D 

Lemma 6.3. For all x E E, P^j-a.s. 

lim — = Ei^(ri). 

n-i>+oo n 

Proof. We can see that 

C 1 " 1 ^"" Ji/ry 1 ^"" 

y£E i=l yeE i=l y£E i=l 

By the ergodic theorem, for any / : £^ i— )■ M, P^-a.s. 

k=0 y£E 

Then we have Pj.-a.s. 



1 n < ^ 



k=0 

Thus, by law of the large numbers, we have the conclusion. D 

Lemma 6.4. For any x E E, F^-a.s. 

lim — ^ = ^'t X , Vy G E, lim — = ^ , , . (6.1) 

t^+oo t Kin) t^+oo t E^(ri) ^ ^ 

Proof. Under P^, the sequence {S^y —S^y)k>i is i.i.d. and by the renewal theorem, 

P^-a.s. 

lim — 



Now, by the strong Markov property of {Xk)k>o 
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Now, since Ey(rj | {Xk)k>o) = ipx,_i{r), we have 






by [H Corollary I 3.6]. Therefore 



Z/j, 



and the proof of the first assertion is complete. Now, it is enough to note that 

ydE 

and this concludes the proof. D 

Proof of Proposition \67T\ Recalling the definition (11. 2p of the empirical measure /it, 
we have ioi y & E 

Nf 
1=1 

By Lemmas 16.31 and 16.41 P^j-a.s. 

hm ^= hm #^ = 1 =^ hm ^^^ = 0. 

On the other hand, by Lemma 16.21 and the law of large numbers, for all bounded 
measurable / : R+ i— )■ M we have P^j-a.s. 



1 _ " _ r°^ 

lim - V T^y f{T^y)= T f{r) i)y{dT) 

"" ^~^ Jo 



and therefore by Lemma [6 



lim - > Tj,v f(Tj,v) = lim rr > Tj,v f(Tj,v) = ^—— / r f{T)ibJdT). 

t^+oo t ^ '^' ■' ^ '^' ' t^+oo t N^ ^ ^^■'^ ^^' E^(ri) Jo ^"^ '^y"^ ^ 

Therefore for all (7 : -Ex ]0, +00 [ bounded and measurable we have 



lim fit{g) = y] ^ f X / Tg{y,T)i)y{dT). 



We prove now the almost sure convergence of the empirical flow Qt{y-, z). We have 

^ N't 

Qt{y,z) = -^ii^x^y^^=z), y,zeE. 



i=l 
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Setting Yi := (X^h_,_j)j^o,...,(/>^ -<i>^y then by the strong Markov property under F^ 
the sequence (Yi)i>i is i.i.d. and its law is equal to the law of {Xj)j^q^ y under P^. 
Then by the law of large numbers, P^^-a.s. 



1 " 



Therefore, by Lemma [6 ■4[ P^^-a.s. 



Nr 



lim Qt(v,z)= lim — ^ ttjt > Ifx « =z) = ^ , ^^n ■ 

i=l ^ ' 

The proof is complete. D 

7. Contraction principles 
Let us consider now the empirical measure of the process (XArJt>o alone, namely 

, , 1 / 1 V ■\ Z — i^'Ni 

Mx)-=jJ l^Xr,,=x)ds=-2_^Tkl^x,_,=x) + Mxn,=x), t > 0. 

We want to obtain a LDP for {711)1 as t — > +00. To this aim, we need the large- 
deviation functional of the empirical measure of the Markov chain {Xk)k>Q: if 

^ n— 1 

Cn{x) = - ^ l(Xfc=x), X e E, 

fc=0 

then the law of (Cn)n satisfies a LDP in the probability measures on E with good 
rate function 



Idv{0= sup ^Cxlogh^^ — =X^Cxlo, 



M6]0,+OoP^g^ \l^yPx,yUy J ^^^ \2^yPx,y'^y ^ 

where u* G ]0, +00 p is the only vector such that 

Cj/ = y]Ca'P*(a;,?/), where p*{x,y):= ^^ 1 

i.e. u* makes C, an invariant measure for p*: see [8[ Theorem IV.6, IV. 7]. 

Proposition 7.1. T/ie /aw 0/ tt^ satisfies a LDP in as t -^ +00 with good rate 
functional 



Ii(7r) = inf ( /w(C) + 5^C. A:(vr./C.) ) 

\ x(^E / 



Proof. For all {f^,Q) G A), let us denote 7r(a;) := /i(x,]0, +00]), x E E. Then 
(7r(a;))j:g£; defines a probability measure on E. The map (/i, Q) 1— )■ vr is continuous 
and we obtain therefore by the contraction principle a LDP for the law of {TTtix))t>o 
as t — 7- +00, where The LDP rate function is given by 

J(7r) = inf{I(/i, Q) : (/i, Q) G Uq, /^(x, ]0, +00]) = 7r(x), x G ^}. 
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In some particular case, the value of J has already been computed, see for instance 
Corollary 5.2 of Duffy and Torrisi [10] . Can we recover the same result with our 
formula? Can we obtain other results beyond this? 



7.1. Computations. We want to compute the infimum of 

^g^J[0,+oo] 






+ e(x)l{oo}(r) 



over all (/i, Q) G Uq such that fi{x, ]0, +cxd]) = 7t{x), for all x E E. We can write 

+ X]C(a;)/^(a;, +oo). 

xeE 

We can suppose that fi{x, +oo) = for all x E E. Now we can compute the 
infimum over {C{x),x G E) of the infimum of \{^,Q) over {fi,Q) G Uq such that 
fi{x,]0, +cxd]) = 7!-(x) andyu(x, 1/r) = ({x), for all x E E. Note that vr is a probability 
measure while ( is positive but not necessarily normalized. Using an analog of [U 
Theorems IV.6 and IV. 7], we obtain that the infimum of 

'p'^{x,yy 



Ip{Q)--= $^g(a:,2/)log('^ 



, p(x,y) 



over Q such that ^ Q{x, y) = ^ Q{y, x) = ({x), for all x G £", is equal to 



sup V Cx log ; 



^yP{x,y)uy 
the unique optimal m* is such that 



and the optimal Q* is 



p{x,y)ul 



Q*{x,y) = Cx^ — 7 T^, x,yeE. 

Indeed, a simple computation shows that 

Notice that in the classical Donsker-Varadhan case, ( should be a probability mea- 
sure, which is not necessarily the case here. 

Therefore we are reduced to compute the infimum of 

lDviC) + J2 [ -M^,rfr)logf^(r) 
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over all ( and /x such that /i(a;, 1/r) = (^, fi{x,W^) = tt^. This is equivalent to 
compute the infimum of 

xeE 

over all C > and (p^ G V(]0, +00 [) such that 0a; (r) = iix/Cx- Now 

inf{H {(p^ I V^o;) : 0^(r) = n^/Cx} = A*(7r^/Cx), 

where A* is the Legendre transform of A^{9) := \og{ipx{e^'^)) , 9 E M.. Then we are 
reduced to 
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